* implemented raw classical instantaneous stdp

* mod to classical stdp

* mod to classical stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* minor mod of syn

* cleaned up stdp syn

* cleaned up stdp syn

* Sync up of main with release (#131)

* Minor nudge to v3.0.1 (#129)

* minor edit to math in hh-lesson doc

* Fix workflow, numpy install, and pytest bug in github action workflows (#117)

* Update pyproject.toml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* minor nudge/cleanup to minor patched version 2.0.1

* minor nudge/cleanup to minor patched version 2.0.3

* Merged back minor doc fix back to main (for syncing purposes) (#119)

* Nudge of release to minor patched version 2.0.3 (#118)

* nudge of doc to 2.0.2 (#115)

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>

* minor edit to math in hh-lesson doc

* Fix workflow, numpy install, and pytest bug in github action workflows (#117)

* Update pyproject.toml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* minor nudge/cleanup to minor patched version 2.0.1

* minor nudge/cleanup to minor patched version 2.0.3

---------

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>

* fixed typo/error in doc evolving_synapses.md

---------

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>

* minor clean-up in model_basics docs

* minor fixes/cleanup of docs

* fixed typo in integration tutorial doc

* updated papers/talk page for ngclearn

* Merging over v3 to main (for roll-out of v3 upgrade) (#125)

* Working v3

* Undid fixed compartemts

Undid the fixed compartments to work with new global constant tracking

* Fixed an execution bug

* ported over quad-lif to v3 - needs testing

* ported over IF/quadLIF cells, minor revision to LIF cell

* Start util cleanup

* refactored/ported RAFCell to v3

* ported over/refactored WTASCell for v3

* wrote successful unit-test of WTASCell

* put back in init-structure/pointers

* fixed minor error in LIFCell, got unit-test for LIFCell to run

* quad-lif test sketched

* sketch of ifcell test

* fixed minor bugs and tests locally pass for if, quad-lif, and lif-cells now, with minor patches to help fun and doc-strings

* refactored raf-cell and test passed

* refactored adex/test passed; minor cleanup in lif, raf, and wtas cells

* refactored fn-cell and test passed

* cleaned up lif, raf, wtas, fn, and quad-lif cells repr method

* refactored and tests passed for izh and h-h cells

* JaxProcess update

* cleaned up dunder repr method, moved to JaxComponent parent; fixed __init__ pointer to tensorstats

* refactored alpha and exp-synapses, tests passed; minor edit to __init__ for synapses

* refactored short-term syn, tests passed - including stp-dense-syn and minor cleanup/edit to synapse __init__

* refactored bcm-syn and test passed

* refactored exp-stdp-syn and passed tests for exp-stdp-syn and trace-stdp-syn

* refactored event-stdp-syn and test passed

* refactored mstdpet-syn and test passed

* refactored stdp-conv-syn/conv-syn and test passed

* refactored and passed test for deconv/stdp-deconv-syn and other minor cleanup for conv/deconv support

* Refactoring neuronal and synaptic components (#123) - merge from fork to v3

* refactoring graded cells

* update refactored models

* update sLIF cell

---------

Co-authored-by: Alex Ororbia <agocse109@gmail.com>

* commented out deprecator in hebb-syn and exp-kernel

* update hebbian synapse

* update hebbian synapse

* working reinforce synapse

* minor edits to exp-kernel/wtas-cell

* update requirements

* refactored conv/deconv-hebb-syn and tests passed

* update hebbian synapse reset bug

* update reset methods

* update patched synapse reset

* add `not self.inputs.targeted and ` to required components. Fixing general `__repr__` bug in `jaxcomponent`

* minor edit to lif/modulated-syn init file

* fixed some minor bugs in rate-coded cells/hebb-syn

* update code

* minor patches to components, including hebb-syn/conv/deconv and reward-cell

* minor patches to components, including hebb-syn/conv/deconv and reward-cell

* update testing for graded neurons and input encoders

* update phasor cell

* update test bernoulli cell and poisson cell

* update components and their related test cases

* fixed monitor bugs from v2, tweaked unit-tests for input-encoders/latency-cell

* update test case for test_sLIFCell.py

* some cleanup

* made revisions to components/clean-up; added back in deprecators

* removed lava sub-module, and removed monitor/base-monitor legacy components

* minor cleanup of inits

* refactored regression module to be compliant with v3

* adjusted sphinx-docs w.r.t. new v3 refactoring

* minor revision to double-exp syn pointing, mods to modeling docs

* updated adex tutorial doc to v3

* revised adex and error-cell neurocog tutorials

* fixed minor issues in input-encoders, further revisions to docs for v3

* revised dyn/chem-syn neurocog doc, cleaned up dynamic syn

* revised fn and hh-cell neurocog docs, added some refs to distribution generator

* revised integration and izh-cell neurocog docs

* revised izh-cell, cleaned-up fn-cell, and revised lif neurocog docs

* revised metrics/plotting neurocog docs

* revised mod/reward-stdp neurocog doc

* revised stp-syn neurocog doc and updated stp-syn to use proper initializer

* revised elements of utils to comply with docs

* revised stdp neurocog doc to v3

* revised traces neurocog tutorial to v3

* cleaned up utils.optim and wrote compliant NAG optim

* cleaned up utils.optim and wrote compliant NAG optim

* cleanup of components, added leaky-noise-cell, minor edits

* revised leaky-noise-cell, wrote its unit test, test-passed

* some revisions/updates to toc/pointer/general tutorial docs

* minor revisions to pyproject/req files

* update reinforce synapse

* update test cases

* implemented in-house gmm, in-built to ngclearn; tested on gaussian mode data

* wrote gmm density estimator tutorial

* patched some tests/syn/neuron components, added sketch of bmm density

* fixed test_laplacianErrorCell and laplace-cell bug

* fixed test_laplacianErrorCell and laplace-cell bug

* made patches to bmm

* updated density tutorial/neurocog doc

* minor edit to gmm/bmm docs

* minor edit to gmm/bmm docs

* cleaned up density structure, use parent mixture class to organize model variations

* cleaned up density structure, use parent mixture class to organize model variations

* added basic exp-mixture to utils.density

* minor edits to emm

* cleaned up mixtures and finished debugging EMM/works on example

* removed old weight_distribution.py, other cleanup/revisions throughout

* minor edit to data-loader

* revised tests to no longer use weight_distribution/revisions throughout

* minor edit to emm doc

* added bic calculation to metric_utils

* fix ratecell  ug of passing unrelated kwargs to parent class

* added calc_update() co-routine to hebbian-syn component

* fix weight init

* integrated rbm/harmonium model-exhibit

* Update __init__.py

Added the config/logging back to the init

* placed pointer to rao-ballard1999 exhibit; updates to docs

* updates to docs/revisions

* removed flag from bernoulli/latency-cells for now; minor edit to doc

* updates to theory doc

* updated history log

* minor clean-up of ngclearn.utils.viz.dim_reduce

* Update jaxComponent.py

Added support for turning off autosave

* update hebbian synapse saving

* update saving and loading utils, making hebbian synapse use these utils for custom optimizer params saving and loading

* minor revisions/polish

* modded docs to include v3 foundations

* updates to init for logging

* Updates to lessons

* final cleanup/polish/update to docs for v3 nudge

* updates to museum doc for v3

* nudged citation file

* minor nudge to docs/files to point to v3

---------

Co-authored-by: Will Gebhardt <will@gebhardts.net>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>
Co-authored-by: Viet Nguyen <alexvn.edu@gmail.com>
Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>

* update to rbm/harmonium doc

* updated leaky-noise-cell to maintain temporal derivative of state

* minor revisons/updates to hebb/dense syn, metric utils

* cleaned-up/revised leaky-noise-cell

* cleaned-up/revised leaky-noise-cell

---------

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>
Co-authored-by: Will Gebhardt <will@gebhardts.net>
Co-authored-by: Viet Nguyen <alexvn.edu@gmail.com>
Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>

* nudge release to v3.0.1

* minor revision of leaky-noise-cell

---------

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>
Co-authored-by: Will Gebhardt <will@gebhardts.net>
Co-authored-by: Viet Nguyen <alexvn.edu@gmail.com>
Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>

* added pointer/stub for ei-rnn song-et-al in museum doc

* update to ei-rnn doc

* update to ei-rnn arch fig

* added log-gaussian initializer to distribution_generator

* bug-fix to log-gaussian func

* Refactor patch utility functions and add doc strings (#136)

* Rao1999 hpc (#135)

* Enhance documentation for predictive coding model

Expanded the documentation for the predictive coding model, detailing the construction of neural and synaptic components, process dynamics, and training procedures.

* Add files via upload

* Update image path for GEC in documentation

* Revise PC model documentation formatting

Updated headings and formatting for clarity in PC model documentation.

* Change header levels for PC model training sections

* Add files via upload

* Update PC model training section with input image details

Added explanation about the input image for the PC model training.

* Update pc_rao_ballard1999.md

* Delete docs/images/museum/hgpc/Patch_input.png

* Add files via upload

* Fix image source and enhance PC model description

Updated image source and adjusted description for clarity.

* Refactor PC model training sections in documentation

Removed the section for training the PC model on the full image and added a reference to it in the patched image section.

* fixed minor errors in pc-rao doc

* made revisions to pc-rao doc

* mod to pc-rao doc

* update to docs

* minor revision to h-h doc-string

* added lkwta utility

* Add retinal ganglion cell input encoder (#137)

* Add RetinalGanglionCell component with filtering methods

Implement RetinalGanglionCell with Gaussian filtering and patch extraction.

* Add RetinalGanglionCell import to input_encoders

* Add RetinalGanglionCell to input encoders

* Enhance filter functions in ganglionCell.py

Refactor Gaussian filter creation and add Difference of Gaussian filter functionality.

* Refactor patch synapse (#138)

* Refactor multi-patch synapse creation and initialization

Refactor _create_multi_patch_synapses function to use n_modules instead of n_sub_models and update weight initialization. Introduce weight masks for synaptic weights.

* Refactor HebbianPatchedSynapse and add attributes

Refactor HebbianPatchedSynapse initialization and add new attributes for post-in and pre-out.

* feat: Integrate MPSSynapse Component (#140)

* feat: integrate MPSSynapse component for compressed synaptic transforms

* style: conform to Google docstrings, move utils, and add unit tests

* feat: implement native learning via evolve method and unit tests

* Fixed MPS Matrix Properties: I fixed the .T transpose bug you interrupted earlier—because self.W10.weights inside an MPSSynapse generates the tensor via an einsum, returning an Array, get() throws an error.

* Fix MPS synapse memory leak by implementing project_backward

* Delete uv.lock

* docs: add academic references and detailed docstrings to MPSSynapse

* sorry, here you go, I loosened the test tolerances to 1e-2 as suggested

---------

Co-authored-by: Alex Ororbia <agocse109@gmail.com>

* integrated working som-synapse into competitive sub-package for synapses

* cleaned up som-syn

* update test code for hebbian patch synapse

* fix SOM Synapse bug

* Flexible batch size (#142)

* Modify reset method to accept batch_size parameter

for flexible test set size

* Modify reset method to accept batch_size parameter

* Refactor RateCell class reset function for flexible batch size

* Refactor GaussianErrorCell class functions for flexible batch size

* flexible batch_size

* cleaned up graded/patched comps with inner batched_reset formulation

* minor clean-up of som-syn

* claned up ganglion-cell, added batched_reset

* minor cleanup

* added working hopfield-syn/modern-hopfield-syn

* update SOM synapse to batchified version

* integrated prototype for vector-quantize memory model/synapse

* wrote/integrated an ART2A synapse model, batch-generalized

* updates to art2a, cleanup of probes

* updates to art2a, cleanup of probes

* added in knn-probe for utils.analysis

* cleaned up vq-synapse

* cleaned up vq-synapse

* tweaked/cleaned-up gaussian-error-cell

* Update JaxProcessesMixin.py

Added automatic jit wrapping which is on by default, add "use_jit=False" to a process to disable

* minor patch fixes, including making .mask a compartment in key syn

* patch to bernoulli/latency and wtas cells

* update reset function of the ganglion cell

Co-authored-by: Copilot <copilot@github.com>

* minor mod to model_utils

* docs now with a few more mods

* Nudge to release of v3.1.0 (#146)

* create release branch

* Dev (#62)

* implemented raw classical instantaneous stdp

* mod to classical stdp

* mod to classical stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* minor mod of syn

* cleaned up stdp syn

* cleaned up stdp syn

---------

Co-authored-by: ago109 <ago109@ist.psu.edu>

* added block diag init to weight dist

* added block diag init to weight dist

* added block diag init to weight dist; with will optimization

* slight extension to rate-cell for tensor-shaping

* slight extension to rate-cell for tensor-shaping

* slight extension to rate-cell for tensor-shaping

* slightly modded bernoulli-cell help to reflect correct compartment names

* nudge to readme for minor version shift to beta2

* nudge to readme for minor version shift to beta2

* mod to docs to prep for nudge to beta2

* nudge correctly to pip version beta3

* generalized rate-cell a bit

* touched up rate-cell further

* minor mod to lif

* updated lif-cell to use units/tags and minor cleanup and edits

* Monitor plot (#66)

* Update base_monitor.py

* added plotting viewed compartments

* added meta-data to rate-cell, input encoders, adex

* fixed minor saving/loading in rate-cell w/ vectorized compartments

* Added auto resolving for monitors (#67)

* fixed surr arg in lif-cell

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* Patched synapses added (#68)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Fix typo in pcn_discrim.md (#69)

* model_utils and rate cell (#70)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* Create hierarchical_sc.md

1

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Delete docs/images/hgpc_network.pdf

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create hgpc

* Delete docs/images/museum/hgpc

* Create d

* Add files via upload

* Delete docs/images/hgpc_model.png

* Delete docs/images/museum/hgpc/d

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Delete docs/images/museum/hgpc/Input_layer.png

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create Generative_PC.md

* Update and rename Generative_PC.md to generative_pc.md

* Update generative_pc.md

* Update generative_pc.md

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update rateCell.py

* Update generative_pc.md

* Create pc-sindy.md

* Update pc-sindy.md

* Update model_utils.py

sine activation function added

* Update model_utils.py

* Update ode_utils.py

jitified

* Delete docs/museum/hierarchical_sc.md

* Delete docs/museum/generative_pc.md

* Delete ngclearn/components/synapses/patched directory

* Update __init__.py

* Add files via upload

ode with scanner added

* Update ode_solver.py

_ removed

* Fix/reorganize feature library (#74)

* Update ode_utils.py

* Update ode_solver.py

rk4 revised and __main__ added

* Delete ngclearn/utils/diffeq/ode_functions.py

* Create odes.py

odes name and structure changed

* Update __init__.py

* Create feature_library.py

* Create __init__.py

* Create base.py

* Delete docs/museum/pc-sindy.md

* Create m.md

* Add files via upload

* Delete docs/images/museum/sindy/m.md

* Add files via upload

* Create sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* fix: correct feature library path and directory name

* Delete ngclearn/utils/dymbolic_dictionary directory

* Update model_utils.py (#78)

* Additions for inhibition stuff

* add sindy documentation for exhibits (#81)

* Add files via upload

* Add files via upload

* Update ode_utils.py (#79)

refactor: delete @partial(jit, static_argnums=(2, )) lines

Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Add patched synapse (#80)

* Update __init__.py

Add point to patched components

* Add patched in __init__.py

Add patched synapses importing

* Add patched synaptic components

* Delete ngclearn/components/synapses/patched/__pycache__ directory

* Update __init__.py

new line characters added

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

New line characters + comments for describing each input vars

* Update patchedSynapse.py

Removed a comment line

* Update hebbianPatchedSynapse.py

remove unused arguments

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

add description for w_mask

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update __init__.py (#83)

* Update __init__.py

typo fixed

* Update staticPatchedSynapse.py

a typo fixed

* Update hebbianPatchedSynapse.py

typo foxed

* Add l1 decay term to update calculation (#84)

* Update hebbianSynapse.py

* update main

update main at the end

* Update hebbianSynapse.py

add regularization argument and w_decay is deprecated.

* Update hebbianSynapse.py

add elastic_net

* Update hebbianSynapse.py

* Update hebbianSynapse.py

* feat NGC module regression (#86)

* feat npc module regression

* Update __init__.py

* Update __init__.py

* Update elastic_net.py

* Update lasso.py

* Update ridge.py

* Update elastic_net.py

* Update ridge.py

* Update lasso.py

* Update odes.py

removed @partial(jit, static_argnums=(0,))

* Update odes.py (#87)

removed @partial(jit, static_argnums=(0,))

* Update odes.py

typo fixed in __main__

* Update __init__.py

add dot

* Update __init__.py

add dot

* Add attribute 'lr' (#90)

* Update elastic_net.py

* add lr as attribute to lasso.py

* add lr as attribute to ridge.py

* refactor w_bound=0. for weights elastic_net.py

deactivated w_bound for weights elastic_net.py

* Update lasso.py

* deactivated w_bound for weights ridge.py

* commit probes/mods to utils to analysis_tools branch

* commit probes/mods to utils to analysis_tools branch

* update documentation

* cleaned up probes/docs for probes

* change heads_dim to attn_dim, and modify the mlp to be as similar as possible to the attentive probing pattern

* in layer normalization or any other Gaussian, standardeviation can never be zero. Additionally, if the subtraction inside the square root goes to zero, the gradient will become NaN. Therefore, adding a clipping is necessary.

* update attentive probe code

* minor tweak to attentive prob code comments

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* minor edits to attn probe

* update attentive probe with input layer norm

* update input layer normalization

* update code to fix nan bug

* minor tweak to attn probe

* cleaned up probes

* cleaned up probes

* cleaned up probes

* cleaned up probes

* generalized dropout in terms of shape

* tweak to atten probe

* tweak to atten probe

* added silu/swish/elu to model_utils

* cleaned up model_utils

* fix bug in attention probe dropout, fix bug in None noise_key passed in the probing jit function, add the spliting of noise_keys to two dropout in two cross attention

* hyperparameter tunning arguments added

* Merging over Dynamics feature branch to main (#92)

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Additions for inhibition stuff

* update to API modeling docs to reflect RAF neuronal cell

---------

Co-authored-by: Alexander Ororbia <agocse109@gmail.com>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* remove unused local variables

* update note

* update model utils

* remove notes

* Update ode utils (#94)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* minor fix to header in diffeq

* Update files with ode_solver (#95)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* Update odes.py

* Update sindy.md

ode_solver to ode_utils

* revised/cleaned up sindy tutorial doc/imgs

* add prior for hebbian patched synapse (#96)

* prior replaced w_decay hebbianPatchedSynapse.py

remove w_decay 
add prior_type and prior_lmbda

* revised typo hebbianSynapse.py

dWweight was typo

* cleaned up doc-strings in odes.py to comply w/ ngc-learn format

* minor tweak to sig-figs printing in probe utils

* add-sigma-to-gaussianErrorCell (#97)

* add-sigma-to-gaussianErrorCell

add not updating scalar variance for gaussian errors

* Update gaussianErrorCell.py

* cleaned up ode_utils, cleaned up gaussian/laplacian cell

* Update gaussianErrorCell.py (#98)

added `and not isinstance(sigma, int)`

* cleaned up gauss/laplace error cells

* integrated bernoulli err-cell

* Major release update merge to main (in prep for 2.0.0 release on release branch/pip) (#99)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* update test cases

---------

Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Major release update (to 2.0.0) (#100)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* major cleanup in prep for merge over to main/prep for major release

* update test cases

* update to require file in docs

---------

Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Major release update merge to main (sync up) (#101)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* major cleanup in prep for merge over to main/prep for major release

* update test cases

* update to require file in docs

---------

Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* update test cases

* Nudging main v2.0.0 to release stage (formal release) (#102)

* generalized rate-cell a bit

* touched up rate-cell further

* minor mod to lif

* updated lif-cell to use units/tags and minor cleanup and edits

* Monitor plot (#66)

* Update base_monitor.py

* added plotting viewed compartments

* added meta-data to rate-cell, input encoders, adex

* fixed minor saving/loading in rate-cell w/ vectorized compartments

* Added auto resolving for monitors (#67)

* fixed surr arg in lif-cell

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* Patched synapses added (#68)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Fix typo in pcn_discrim.md (#69)

* model_utils and rate cell (#70)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* Create hierarchical_sc.md

1

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Delete docs/images/hgpc_network.pdf

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create hgpc

* Delete docs/images/museum/hgpc

* Create d

* Add files via upload

* Delete docs/images/hgpc_model.png

* Delete docs/images/museum/hgpc/d

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Delete docs/images/museum/hgpc/Input_layer.png

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create Generative_PC.md

* Update and rename Generative_PC.md to generative_pc.md

* Update generative_pc.md

* Update generative_pc.md

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update rateCell.py

* Update generative_pc.md

* Create pc-sindy.md

* Update pc-sindy.md

* Update model_utils.py

sine activation function added

* Update model_utils.py

* Update ode_utils.py

jitified

* Delete docs/museum/hierarchical_sc.md

* Delete docs/museum/generative_pc.md

* Delete ngclearn/components/synapses/patched directory

* Update __init__.py

* Add files via upload

ode with scanner added

* Update ode_solver.py

_ removed

* Fix/reorganize feature library (#74)

* Update ode_utils.py

* Update ode_solver.py

rk4 revised and __main__ added

* Delete ngclearn/utils/diffeq/ode_functions.py

* Create odes.py

odes name and structure changed

* Update __init__.py

* Create feature_library.py

* Create __init__.py

* Create base.py

* Delete docs/museum/pc-sindy.md

* Create m.md

* Add files via upload

* Delete docs/images/museum/sindy/m.md

* Add files via upload

* Create sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* fix: correct feature library path and directory name

* Delete ngclearn/utils/dymbolic_dictionary directory

* Update model_utils.py (#78)

* Additions for inhibition stuff

* add sindy documentation for exhibits (#81)

* Add files via upload

* Add files via upload

* Update ode_utils.py (#79)

refactor: delete @partial(jit, static_argnums=(2, )) lines

Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Add patched synapse (#80)

* Update __init__.py

Add point to patched components

* Add patched in __init__.py

Add patched synapses importing

* Add patched synaptic components

* Delete ngclearn/components/synapses/patched/__pycache__ directory

* Update __init__.py

new line characters added

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

New line characters + comments for describing each input vars

* Update patchedSynapse.py

Removed a comment line

* Update hebbianPatchedSynapse.py

remove unused arguments

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

add description for w_mask

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update __init__.py (#83)

* Update __init__.py

typo fixed

* Update staticPatchedSynapse.py

a typo fixed

* Update hebbianPatchedSynapse.py

typo foxed

* Add l1 decay term to update calculation (#84)

* Update hebbianSynapse.py

* update main

update main at the end

* Update hebbianSynapse.py

add regularization argument and w_decay is deprecated.

* Update hebbianSynapse.py

add elastic_net

* Update hebbianSynapse.py

* Update hebbianSynapse.py

* feat NGC module regression (#86)

* feat npc module regression

* Update __init__.py

* Update __init__.py

* Update elastic_net.py

* Update lasso.py

* Update ridge.py

* Update elastic_net.py

* Update ridge.py

* Update lasso.py

* Update odes.py

removed @partial(jit, static_argnums=(0,))

* Update odes.py (#87)

removed @partial(jit, static_argnums=(0,))

* Update odes.py

typo fixed in __main__

* Update __init__.py

add dot

* Update __init__.py

add dot

* Add attribute 'lr' (#90)

* Update elastic_net.py

* add lr as attribute to lasso.py

* add lr as attribute to ridge.py

* refactor w_bound=0. for weights elastic_net.py

deactivated w_bound for weights elastic_net.py

* Update lasso.py

* deactivated w_bound for weights ridge.py

* commit probes/mods to utils to analysis_tools branch

* commit probes/mods to utils to analysis_tools branch

* update documentation

* cleaned up probes/docs for probes

* change heads_dim to attn_dim, and modify the mlp to be as similar as possible to the attentive probing pattern

* in layer normalization or any other Gaussian, standardeviation can never be zero. Additionally, if the subtraction inside the square root goes to zero, the gradient will become NaN. Therefore, adding a clipping is necessary.

* update attentive probe code

* minor tweak to attentive prob code comments

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* minor edits to attn probe

* update attentive probe with input layer norm

* update input layer normalization

* update code to fix nan bug

* minor tweak to attn probe

* cleaned up probes

* cleaned up probes

* cleaned up probes

* cleaned up probes

* generalized dropout in terms of shape

* tweak to atten probe

* tweak to atten probe

* added silu/swish/elu to model_utils

* cleaned up model_utils

* fix bug in attention probe dropout, fix bug in None noise_key passed in the probing jit function, add the spliting of noise_keys to two dropout in two cross attention

* hyperparameter tunning arguments added

* Merging over Dynamics feature branch to main (#92)

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Additions for inhibition stuff

* update to API modeling docs to reflect RAF neuronal cell

---------

Co-authored-by: Alexander Ororbia <agocse109@gmail.com>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* remove unused local variables

* update note

* update model utils

* remove notes

* Update ode utils (#94)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* minor fix to header in diffeq

* Update files with ode_solver (#95)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* Update odes.py

* Update sindy.md

ode_solver to ode_utils

* revised/cleaned up sindy tutorial doc/imgs

* add prior for hebbian patched synapse (#96)

* prior replaced w_decay hebbianPatchedSynapse.py

remove w_decay 
add prior_type and prior_lmbda

* revised typo hebbianSynapse.py

dWweight was typo

* cleaned up doc-strings in odes.py to comply w/ ngc-learn format

* minor tweak to sig-figs printing in probe utils

* add-sigma-to-gaussianErrorCell (#97)

* add-sigma-to-gaussianErrorCell

add not updating scalar variance for gaussian errors

* Update gaussianErrorCell.py

* cleaned up ode_utils, cleaned up gaussian/laplacian cell

* Update gaussianErrorCell.py (#98)

added `and not isinstance(sigma, int)`

* cleaned up gauss/laplace error cells

* integrated bernoulli err-cell

* Major release update merge to main (in prep for 2.0.0 release on release branch/pip) (#99)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* update test cases

---------

Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Major release update (to 2.0.0) (#100)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked…

* Final nudge to v3.1.0 (#149)

* implemented raw classical instantaneous stdp

* mod to classical stdp

* mod to classical stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* minor mod of syn

* cleaned up stdp syn

* cleaned up stdp syn

* Sync up of main with release (#131)

* Minor nudge to v3.0.1 (#129)

* minor edit to math in hh-lesson doc

* Fix workflow, numpy install, and pytest bug in github action workflows (#117)

* Update pyproject.toml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* minor nudge/cleanup to minor patched version 2.0.1

* minor nudge/cleanup to minor patched version 2.0.3

* Merged back minor doc fix back to main (for syncing purposes) (#119)

* Nudge of release to minor patched version 2.0.3 (#118)

* nudge of doc to 2.0.2 (#115)

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>

* minor edit to math in hh-lesson doc

* Fix workflow, numpy install, and pytest bug in github action workflows (#117)

* Update pyproject.toml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* Update python-package-conda.yml

* minor nudge/cleanup to minor patched version 2.0.1

* minor nudge/cleanup to minor patched version 2.0.3

---------

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>

* fixed typo/error in doc evolving_synapses.md

---------

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>

* minor clean-up in model_basics docs

* minor fixes/cleanup of docs

* fixed typo in integration tutorial doc

* updated papers/talk page for ngclearn

* Merging over v3 to main (for roll-out of v3 upgrade) (#125)

* Working v3

* Undid fixed compartemts

Undid the fixed compartments to work with new global constant tracking

* Fixed an execution bug

* ported over quad-lif to v3 - needs testing

* ported over IF/quadLIF cells, minor revision to LIF cell

* Start util cleanup

* refactored/ported RAFCell to v3

* ported over/refactored WTASCell for v3

* wrote successful unit-test of WTASCell

* put back in init-structure/pointers

* fixed minor error in LIFCell, got unit-test for LIFCell to run

* quad-lif test sketched

* sketch of ifcell test

* fixed minor bugs and tests locally pass for if, quad-lif, and lif-cells now, with minor patches to help fun and doc-strings

* refactored raf-cell and test passed

* refactored adex/test passed; minor cleanup in lif, raf, and wtas cells

* refactored fn-cell and test passed

* cleaned up lif, raf, wtas, fn, and quad-lif cells repr method

* refactored and tests passed for izh and h-h cells

* JaxProcess update

* cleaned up dunder repr method, moved to JaxComponent parent; fixed __init__ pointer to tensorstats

* refactored alpha and exp-synapses, tests passed; minor edit to __init__ for synapses

* refactored short-term syn, tests passed - including stp-dense-syn and minor cleanup/edit to synapse __init__

* refactored bcm-syn and test passed

* refactored exp-stdp-syn and passed tests for exp-stdp-syn and trace-stdp-syn

* refactored event-stdp-syn and test passed

* refactored mstdpet-syn and test passed

* refactored stdp-conv-syn/conv-syn and test passed

* refactored and passed test for deconv/stdp-deconv-syn and other minor cleanup for conv/deconv support

* Refactoring neuronal and synaptic components (#123) - merge from fork to v3

* refactoring graded cells

* update refactored models

* update sLIF cell

---------

Co-authored-by: Alex Ororbia <agocse109@gmail.com>

* commented out deprecator in hebb-syn and exp-kernel

* update hebbian synapse

* update hebbian synapse

* working reinforce synapse

* minor edits to exp-kernel/wtas-cell

* update requirements

* refactored conv/deconv-hebb-syn and tests passed

* update hebbian synapse reset bug

* update reset methods

* update patched synapse reset

* add `not self.inputs.targeted and ` to required components. Fixing general `__repr__` bug in `jaxcomponent`

* minor edit to lif/modulated-syn init file

* fixed some minor bugs in rate-coded cells/hebb-syn

* update code

* minor patches to components, including hebb-syn/conv/deconv and reward-cell

* minor patches to components, including hebb-syn/conv/deconv and reward-cell

* update testing for graded neurons and input encoders

* update phasor cell

* update test bernoulli cell and poisson cell

* update components and their related test cases

* fixed monitor bugs from v2, tweaked unit-tests for input-encoders/latency-cell

* update test case for test_sLIFCell.py

* some cleanup

* made revisions to components/clean-up; added back in deprecators

* removed lava sub-module, and removed monitor/base-monitor legacy components

* minor cleanup of inits

* refactored regression module to be compliant with v3

* adjusted sphinx-docs w.r.t. new v3 refactoring

* minor revision to double-exp syn pointing, mods to modeling docs

* updated adex tutorial doc to v3

* revised adex and error-cell neurocog tutorials

* fixed minor issues in input-encoders, further revisions to docs for v3

* revised dyn/chem-syn neurocog doc, cleaned up dynamic syn

* revised fn and hh-cell neurocog docs, added some refs to distribution generator

* revised integration and izh-cell neurocog docs

* revised izh-cell, cleaned-up fn-cell, and revised lif neurocog docs

* revised metrics/plotting neurocog docs

* revised mod/reward-stdp neurocog doc

* revised stp-syn neurocog doc and updated stp-syn to use proper initializer

* revised elements of utils to comply with docs

* revised stdp neurocog doc to v3

* revised traces neurocog tutorial to v3

* cleaned up utils.optim and wrote compliant NAG optim

* cleaned up utils.optim and wrote compliant NAG optim

* cleanup of components, added leaky-noise-cell, minor edits

* revised leaky-noise-cell, wrote its unit test, test-passed

* some revisions/updates to toc/pointer/general tutorial docs

* minor revisions to pyproject/req files

* update reinforce synapse

* update test cases

* implemented in-house gmm, in-built to ngclearn; tested on gaussian mode data

* wrote gmm density estimator tutorial

* patched some tests/syn/neuron components, added sketch of bmm density

* fixed test_laplacianErrorCell and laplace-cell bug

* fixed test_laplacianErrorCell and laplace-cell bug

* made patches to bmm

* updated density tutorial/neurocog doc

* minor edit to gmm/bmm docs

* minor edit to gmm/bmm docs

* cleaned up density structure, use parent mixture class to organize model variations

* cleaned up density structure, use parent mixture class to organize model variations

* added basic exp-mixture to utils.density

* minor edits to emm

* cleaned up mixtures and finished debugging EMM/works on example

* removed old weight_distribution.py, other cleanup/revisions throughout

* minor edit to data-loader

* revised tests to no longer use weight_distribution/revisions throughout

* minor edit to emm doc

* added bic calculation to metric_utils

* fix ratecell  ug of passing unrelated kwargs to parent class

* added calc_update() co-routine to hebbian-syn component

* fix weight init

* integrated rbm/harmonium model-exhibit

* Update __init__.py

Added the config/logging back to the init

* placed pointer to rao-ballard1999 exhibit; updates to docs

* updates to docs/revisions

* removed flag from bernoulli/latency-cells for now; minor edit to doc

* updates to theory doc

* updated history log

* minor clean-up of ngclearn.utils.viz.dim_reduce

* Update jaxComponent.py

Added support for turning off autosave

* update hebbian synapse saving

* update saving and loading utils, making hebbian synapse use these utils for custom optimizer params saving and loading

* minor revisions/polish

* modded docs to include v3 foundations

* updates to init for logging

* Updates to lessons

* final cleanup/polish/update to docs for v3 nudge

* updates to museum doc for v3

* nudged citation file

* minor nudge to docs/files to point to v3

---------

Co-authored-by: Will Gebhardt <will@gebhardts.net>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>
Co-authored-by: Viet Nguyen <alexvn.edu@gmail.com>
Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>

* update to rbm/harmonium doc

* updated leaky-noise-cell to maintain temporal derivative of state

* minor revisons/updates to hebb/dense syn, metric utils

* cleaned-up/revised leaky-noise-cell

* cleaned-up/revised leaky-noise-cell

---------

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>
Co-authored-by: Will Gebhardt <will@gebhardts.net>
Co-authored-by: Viet Nguyen <alexvn.edu@gmail.com>
Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>

* nudge release to v3.0.1

* minor revision of leaky-noise-cell

---------

Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Viet Dung Nguyen <60036798+rxng8@users.noreply.github.com>
Co-authored-by: Will Gebhardt <will@gebhardts.net>
Co-authored-by: Viet Nguyen <alexvn.edu@gmail.com>
Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>

* added pointer/stub for ei-rnn song-et-al in museum doc

* update to ei-rnn doc

* update to ei-rnn arch fig

* added log-gaussian initializer to distribution_generator

* bug-fix to log-gaussian func

* Refactor patch utility functions and add doc strings (#136)

* Rao1999 hpc (#135)

* Enhance documentation for predictive coding model

Expanded the documentation for the predictive coding model, detailing the construction of neural and synaptic components, process dynamics, and training procedures.

* Add files via upload

* Update image path for GEC in documentation

* Revise PC model documentation formatting

Updated headings and formatting for clarity in PC model documentation.

* Change header levels for PC model training sections

* Add files via upload

* Update PC model training section with input image details

Added explanation about the input image for the PC model training.

* Update pc_rao_ballard1999.md

* Delete docs/images/museum/hgpc/Patch_input.png

* Add files via upload

* Fix image source and enhance PC model description

Updated image source and adjusted description for clarity.

* Refactor PC model training sections in documentation

Removed the section for training the PC model on the full image and added a reference to it in the patched image section.

* fixed minor errors in pc-rao doc

* made revisions to pc-rao doc

* mod to pc-rao doc

* update to docs

* minor revision to h-h doc-string

* added lkwta utility

* Add retinal ganglion cell input encoder (#137)

* Add RetinalGanglionCell component with filtering methods

Implement RetinalGanglionCell with Gaussian filtering and patch extraction.

* Add RetinalGanglionCell import to input_encoders

* Add RetinalGanglionCell to input encoders

* Enhance filter functions in ganglionCell.py

Refactor Gaussian filter creation and add Difference of Gaussian filter functionality.

* Refactor patch synapse (#138)

* Refactor multi-patch synapse creation and initialization

Refactor _create_multi_patch_synapses function to use n_modules instead of n_sub_models and update weight initialization. Introduce weight masks for synaptic weights.

* Refactor HebbianPatchedSynapse and add attributes

Refactor HebbianPatchedSynapse initialization and add new attributes for post-in and pre-out.

* feat: Integrate MPSSynapse Component (#140)

* feat: integrate MPSSynapse component for compressed synaptic transforms

* style: conform to Google docstrings, move utils, and add unit tests

* feat: implement native learning via evolve method and unit tests

* Fixed MPS Matrix Properties: I fixed the .T transpose bug you interrupted earlier—because self.W10.weights inside an MPSSynapse generates the tensor via an einsum, returning an Array, get() throws an error.

* Fix MPS synapse memory leak by implementing project_backward

* Delete uv.lock

* docs: add academic references and detailed docstrings to MPSSynapse

* sorry, here you go, I loosened the test tolerances to 1e-2 as suggested

---------

Co-authored-by: Alex Ororbia <agocse109@gmail.com>

* integrated working som-synapse into competitive sub-package for synapses

* cleaned up som-syn

* update test code for hebbian patch synapse

* fix SOM Synapse bug

* Flexible batch size (#142)

* Modify reset method to accept batch_size parameter

for flexible test set size

* Modify reset method to accept batch_size parameter

* Refactor RateCell class reset function for flexible batch size

* Refactor GaussianErrorCell class functions for flexible batch size

* flexible batch_size

* cleaned up graded/patched comps with inner batched_reset formulation

* minor clean-up of som-syn

* claned up ganglion-cell, added batched_reset

* minor cleanup

* added working hopfield-syn/modern-hopfield-syn

* update SOM synapse to batchified version

* integrated prototype for vector-quantize memory model/synapse

* wrote/integrated an ART2A synapse model, batch-generalized

* updates to art2a, cleanup of probes

* updates to art2a, cleanup of probes

* added in knn-probe for utils.analysis

* cleaned up vq-synapse

* cleaned up vq-synapse

* tweaked/cleaned-up gaussian-error-cell

* Update JaxProcessesMixin.py

Added automatic jit wrapping which is on by default, add "use_jit=False" to a process to disable

* minor patch fixes, including making .mask a compartment in key syn

* patch to bernoulli/latency and wtas cells

* update reset function of the ganglion cell

Co-authored-by: Copilot <copilot@github.com>

* minor mod to model_utils

* docs now with a few more mods

* Nudge to release of v3.1.0 (#146)

* create release branch

* Dev (#62)

* implemented raw classical instantaneous stdp

* mod to classical stdp

* mod to classical stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* mod to stdp syn

* minor mod of syn

* cleaned up stdp syn

* cleaned up stdp syn

---------

Co-authored-by: ago109 <ago109@ist.psu.edu>

* added block diag init to weight dist

* added block diag init to weight dist

* added block diag init to weight dist; with will optimization

* slight extension to rate-cell for tensor-shaping

* slight extension to rate-cell for tensor-shaping

* slight extension to rate-cell for tensor-shaping

* slightly modded bernoulli-cell help to reflect correct compartment names

* nudge to readme for minor version shift to beta2

* nudge to readme for minor version shift to beta2

* mod to docs to prep for nudge to beta2

* nudge correctly to pip version beta3

* generalized rate-cell a bit

* touched up rate-cell further

* minor mod to lif

* updated lif-cell to use units/tags and minor cleanup and edits

* Monitor plot (#66)

* Update base_monitor.py

* added plotting viewed compartments

* added meta-data to rate-cell, input encoders, adex

* fixed minor saving/loading in rate-cell w/ vectorized compartments

* Added auto resolving for monitors (#67)

* fixed surr arg in lif-cell

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* Patched synapses added (#68)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Fix typo in pcn_discrim.md (#69)

* model_utils and rate cell (#70)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* Create hierarchical_sc.md

1

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Delete docs/images/hgpc_network.pdf

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create hgpc

* Delete docs/images/museum/hgpc

* Create d

* Add files via upload

* Delete docs/images/hgpc_model.png

* Delete docs/images/museum/hgpc/d

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Delete docs/images/museum/hgpc/Input_layer.png

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create Generative_PC.md

* Update and rename Generative_PC.md to generative_pc.md

* Update generative_pc.md

* Update generative_pc.md

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update rateCell.py

* Update generative_pc.md

* Create pc-sindy.md

* Update pc-sindy.md

* Update model_utils.py

sine activation function added

* Update model_utils.py

* Update ode_utils.py

jitified

* Delete docs/museum/hierarchical_sc.md

* Delete docs/museum/generative_pc.md

* Delete ngclearn/components/synapses/patched directory

* Update __init__.py

* Add files via upload

ode with scanner added

* Update ode_solver.py

_ removed

* Fix/reorganize feature library (#74)

* Update ode_utils.py

* Update ode_solver.py

rk4 revised and __main__ added

* Delete ngclearn/utils/diffeq/ode_functions.py

* Create odes.py

odes name and structure changed

* Update __init__.py

* Create feature_library.py

* Create __init__.py

* Create base.py

* Delete docs/museum/pc-sindy.md

* Create m.md

* Add files via upload

* Delete docs/images/museum/sindy/m.md

* Add files via upload

* Create sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* fix: correct feature library path and directory name

* Delete ngclearn/utils/dymbolic_dictionary directory

* Update model_utils.py (#78)

* Additions for inhibition stuff

* add sindy documentation for exhibits (#81)

* Add files via upload

* Add files via upload

* Update ode_utils.py (#79)

refactor: delete @partial(jit, static_argnums=(2, )) lines

Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Add patched synapse (#80)

* Update __init__.py

Add point to patched components

* Add patched in __init__.py

Add patched synapses importing

* Add patched synaptic components

* Delete ngclearn/components/synapses/patched/__pycache__ directory

* Update __init__.py

new line characters added

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

New line characters + comments for describing each input vars

* Update patchedSynapse.py

Removed a comment line

* Update hebbianPatchedSynapse.py

remove unused arguments

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

add description for w_mask

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update __init__.py (#83)

* Update __init__.py

typo fixed

* Update staticPatchedSynapse.py

a typo fixed

* Update hebbianPatchedSynapse.py

typo foxed

* Add l1 decay term to update calculation (#84)

* Update hebbianSynapse.py

* update main

update main at the end

* Update hebbianSynapse.py

add regularization argument and w_decay is deprecated.

* Update hebbianSynapse.py

add elastic_net

* Update hebbianSynapse.py

* Update hebbianSynapse.py

* feat NGC module regression (#86)

* feat npc module regression

* Update __init__.py

* Update __init__.py

* Update elastic_net.py

* Update lasso.py

* Update ridge.py

* Update elastic_net.py

* Update ridge.py

* Update lasso.py

* Update odes.py

removed @partial(jit, static_argnums=(0,))

* Update odes.py (#87)

removed @partial(jit, static_argnums=(0,))

* Update odes.py

typo fixed in __main__

* Update __init__.py

add dot

* Update __init__.py

add dot

* Add attribute 'lr' (#90)

* Update elastic_net.py

* add lr as attribute to lasso.py

* add lr as attribute to ridge.py

* refactor w_bound=0. for weights elastic_net.py

deactivated w_bound for weights elastic_net.py

* Update lasso.py

* deactivated w_bound for weights ridge.py

* commit probes/mods to utils to analysis_tools branch

* commit probes/mods to utils to analysis_tools branch

* update documentation

* cleaned up probes/docs for probes

* change heads_dim to attn_dim, and modify the mlp to be as similar as possible to the attentive probing pattern

* in layer normalization or any other Gaussian, standardeviation can never be zero. Additionally, if the subtraction inside the square root goes to zero, the gradient will become NaN. Therefore, adding a clipping is necessary.

* update attentive probe code

* minor tweak to attentive prob code comments

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* minor edits to attn probe

* update attentive probe with input layer norm

* update input layer normalization

* update code to fix nan bug

* minor tweak to attn probe

* cleaned up probes

* cleaned up probes

* cleaned up probes

* cleaned up probes

* generalized dropout in terms of shape

* tweak to atten probe

* tweak to atten probe

* added silu/swish/elu to model_utils

* cleaned up model_utils

* fix bug in attention probe dropout, fix bug in None noise_key passed in the probing jit function, add the spliting of noise_keys to two dropout in two cross attention

* hyperparameter tunning arguments added

* Merging over Dynamics feature branch to main (#92)

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Additions for inhibition stuff

* update to API modeling docs to reflect RAF neuronal cell

---------

Co-authored-by: Alexander Ororbia <agocse109@gmail.com>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* remove unused local variables

* update note

* update model utils

* remove notes

* Update ode utils (#94)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* minor fix to header in diffeq

* Update files with ode_solver (#95)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* Update odes.py

* Update sindy.md

ode_solver to ode_utils

* revised/cleaned up sindy tutorial doc/imgs

* add prior for hebbian patched synapse (#96)

* prior replaced w_decay hebbianPatchedSynapse.py

remove w_decay 
add prior_type and prior_lmbda

* revised typo hebbianSynapse.py

dWweight was typo

* cleaned up doc-strings in odes.py to comply w/ ngc-learn format

* minor tweak to sig-figs printing in probe utils

* add-sigma-to-gaussianErrorCell (#97)

* add-sigma-to-gaussianErrorCell

add not updating scalar variance for gaussian errors

* Update gaussianErrorCell.py

* cleaned up ode_utils, cleaned up gaussian/laplacian cell

* Update gaussianErrorCell.py (#98)

added `and not isinstance(sigma, int)`

* cleaned up gauss/laplace error cells

* integrated bernoulli err-cell

* Major release update merge to main (in prep for 2.0.0 release on release branch/pip) (#99)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* update test cases

---------

Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Major release update (to 2.0.0) (#100)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* major cleanup in prep for merge over to main/prep for major release

* update test cases

* update to require file in docs

---------

Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Major release update merge to main (sync up) (#101)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* major cleanup in prep for merge over to main/prep for major release

* update test cases

* update to require file in docs

---------

Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* update test cases

* Nudging main v2.0.0 to release stage (formal release) (#102)

* generalized rate-cell a bit

* touched up rate-cell further

* minor mod to lif

* updated lif-cell to use units/tags and minor cleanup and edits

* Monitor plot (#66)

* Update base_monitor.py

* added plotting viewed compartments

* added meta-data to rate-cell, input encoders, adex

* fixed minor saving/loading in rate-cell w/ vectorized compartments

* Added auto resolving for monitors (#67)

* fixed surr arg in lif-cell

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* Patched synapses added (#68)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Fix typo in pcn_discrim.md (#69)

* model_utils and rate cell (#70)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* Create hierarchical_sc.md

1

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Delete docs/images/hgpc_network.pdf

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create hgpc

* Delete docs/images/museum/hgpc

* Create d

* Add files via upload

* Delete docs/images/hgpc_model.png

* Delete docs/images/museum/hgpc/d

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Delete docs/images/museum/hgpc/Input_layer.png

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create Generative_PC.md

* Update and rename Generative_PC.md to generative_pc.md

* Update generative_pc.md

* Update generative_pc.md

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update rateCell.py

* Update generative_pc.md

* Create pc-sindy.md

* Update pc-sindy.md

* Update model_utils.py

sine activation function added

* Update model_utils.py

* Update ode_utils.py

jitified

* Delete docs/museum/hierarchical_sc.md

* Delete docs/museum/generative_pc.md

* Delete ngclearn/components/synapses/patched directory

* Update __init__.py

* Add files via upload

ode with scanner added

* Update ode_solver.py

_ removed

* Fix/reorganize feature library (#74)

* Update ode_utils.py

* Update ode_solver.py

rk4 revised and __main__ added

* Delete ngclearn/utils/diffeq/ode_functions.py

* Create odes.py

odes name and structure changed

* Update __init__.py

* Create feature_library.py

* Create __init__.py

* Create base.py

* Delete docs/museum/pc-sindy.md

* Create m.md

* Add files via upload

* Delete docs/images/museum/sindy/m.md

* Add files via upload

* Create sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* fix: correct feature library path and directory name

* Delete ngclearn/utils/dymbolic_dictionary directory

* Update model_utils.py (#78)

* Additions for inhibition stuff

* add sindy documentation for exhibits (#81)

* Add files via upload

* Add files via upload

* Update ode_utils.py (#79)

refactor: delete @partial(jit, static_argnums=(2, )) lines

Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Add patched synapse (#80)

* Update __init__.py

Add point to patched components

* Add patched in __init__.py

Add patched synapses importing

* Add patched synaptic components

* Delete ngclearn/components/synapses/patched/__pycache__ directory

* Update __init__.py

new line characters added

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

New line characters + comments for describing each input vars

* Update patchedSynapse.py

Removed a comment line

* Update hebbianPatchedSynapse.py

remove unused arguments

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

add description for w_mask

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update __init__.py (#83)

* Update __init__.py

typo fixed

* Update staticPatchedSynapse.py

a typo fixed

* Update hebbianPatchedSynapse.py

typo foxed

* Add l1 decay term to update calculation (#84)

* Update hebbianSynapse.py

* update main

update main at the end

* Update hebbianSynapse.py

add regularization argument and w_decay is deprecated.

* Update hebbianSynapse.py

add elastic_net

* Update hebbianSynapse.py

* Update hebbianSynapse.py

* feat NGC module regression (#86)

* feat npc module regression

* Update __init__.py

* Update __init__.py

* Update elastic_net.py

* Update lasso.py

* Update ridge.py

* Update elastic_net.py

* Update ridge.py

* Update lasso.py

* Update odes.py

removed @partial(jit, static_argnums=(0,))

* Update odes.py (#87)

removed @partial(jit, static_argnums=(0,))

* Update odes.py

typo fixed in __main__

* Update __init__.py

add dot

* Update __init__.py

add dot

* Add attribute 'lr' (#90)

* Update elastic_net.py

* add lr as attribute to lasso.py

* add lr as attribute to ridge.py

* refactor w_bound=0. for weights elastic_net.py

deactivated w_bound for weights elastic_net.py

* Update lasso.py

* deactivated w_bound for weights ridge.py

* commit probes/mods to utils to analysis_tools branch

* commit probes/mods to utils to analysis_tools branch

* update documentation

* cleaned up probes/docs for probes

* change heads_dim to attn_dim, and modify the mlp to be as similar as possible to the attentive probing pattern

* in layer normalization or any other Gaussian, standardeviation can never be zero. Additionally, if the subtraction inside the square root goes to zero, the gradient will become NaN. Therefore, adding a clipping is necessary.

* update attentive probe code

* minor tweak to attentive prob code comments

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* minor edits to attn probe

* update attentive probe with input layer norm

* update input layer normalization

* update code to fix nan bug

* minor tweak to attn probe

* cleaned up probes

* cleaned up probes

* cleaned up probes

* cleaned up probes

* generalized dropout in terms of shape

* tweak to atten probe

* tweak to atten probe

* added silu/swish/elu to model_utils

* cleaned up model_utils

* fix bug in attention probe dropout, fix bug in None noise_key passed in the probing jit function, add the spliting of noise_keys to two dropout in two cross attention

* hyperparameter tunning arguments added

* Merging over Dynamics feature branch to main (#92)

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Additions for inhibition stuff

* update to API modeling docs to reflect RAF neuronal cell

---------

Co-authored-by: Alexander Ororbia <agocse109@gmail.com>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* remove unused local variables

* update note

* update model utils

* remove notes

* Update ode utils (#94)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* minor fix to header in diffeq

* Update files with ode_solver (#95)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* Update odes.py

* Update sindy.md

ode_solver to ode_utils

* revised/cleaned up sindy tutorial doc/imgs

* add prior for hebbian patched synapse (#96)

* prior replaced w_decay hebbianPatchedSynapse.py

remove w_decay 
add prior_type and prior_lmbda

* revised typo hebbianSynapse.py

dWweight was typo

* cleaned up doc-strings in odes.py to comply w/ ngc-learn format

* minor tweak to sig-figs printing in probe utils

* add-sigma-to-gaussianErrorCell (#97)

* add-sigma-to-gaussianErrorCell

add not updating scalar variance for gaussian errors

* Update gaussianErrorCell.py

* cleaned up ode_utils, cleaned up gaussian/laplacian cell

* Update gaussianErrorCell.py (#98)

added `and not isinstance(sigma, int)`

* cleaned up gauss/laplace error cells

* integrated bernoulli err-cell

* Major release update merge to main (in prep for 2.0.0 release on release branch/pip) (#99)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* update test cases

---------

Co-authored-by: Viet Dung Nguyen <vietdungnguyen233@gmail.com>
Co-authored-by: Alexander Ororbia <ago@hal3.cs.rit.edu>
Co-authored-by: Will Gebhardt <will@gebhardts.net>

* Major release update (to 2.0.0) (#100)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to ref…